Area of focus: Hand Gesture Recognition Technology
The area of focus for this project will mainly be focusing on the hand gesture recognition technology.
Research Background on Area of focus:
As we already know, technology is constantly changing and advancing. It affects and changes the way we used to live and behave. And just in the recent years, we started to see a trend of utilizing the location and movements of our hands and fingers into the technology to perform different types of actions that we have never done of before. For instance, “Gest” would be one of the examples that demonstrate how our hand gestures are being combined with the technology and becomes a wearable device that allow us to control our computer display or mobile device without the need of touching our keyboard or mouse.
Discovery to Ideation Phase to Concept :
My initial ideation for this project was inspired by the "Gest" that is mentioned above as well as the finger movements of human hand during piano playing, with every finger playing its role in affecting the final output of sound or music. Therefore, what I had in mind at first was related to some kinds of glove that allow us to play the piano keys without needing the actual piano. However, I soon realized that by having only the piano sound as the output sets limits on the types of user that will be using it. And this is the point where the concept of this project gets modifed, and becomes what it is right now, which is a hand gesture controlled music player that uses motion sensor and color sensors to output a mixture of sound/music. The motion sensor will be to used detect if the hand has any turning or shaking action, if there is, then the volume of the music will get increased or decreased based on the detected rate of acceleration in 3D space, while the color sensor that are attached onto each fingertip will be used to detect the colors of the objects that the fingertips are pointing toward to, therefore based on the colors within surrounding environment of the user, as well as the angle or direction the fingertips are facing to, it will output different types of instrument sounds that are remixed with other instrument sounds.
Sketches:
Interaction model:
Why This Might Be A Good Project?
This might be a good project, because the purpose of it is to simply play with our hands to create music. Using our hand and finger motions and the color of our surrounding as the core inputs for this project provides almost a infinite number of combinations of music. How much control you have over your finger also play in a role in determing the final output sound, like the different angles that your fingers can bend to obtain the right color from your surroundding environment will influence the way that the music/sound put together, therefore, this project allows users to explore their capability and limits of playing with their hands.
Equipment Needed:
Adafruit's FLORA
VS1053 Codec
MicroSD Breakout - MP3/WAV/MIDI/OGG Play + Record
Stereo 3.7W Class D Audio Amplifier - MAX98306
FLORA color sensor
FLORA accelerometer sensor
Skills Needed:
Programming Skills
Sound Editing Skills
Sewing skills
Reference (Adafruit Tutorials):
https://www.youtube.com/watch?v=RaNr-NUJx3s
https://www.youtube.com/watch?v=crkJLCR8agA
